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LONG-TERM  GOALS 

Growing  concerns,  that  aquatic  noise  produced  during  naval  exercises  and  offshore  seismic  surveys 
may  be  harmful  to  marine  mammals,  have  led  an  increasing  number  of  regulating  agencies  to  request 
mitigation  measures  when  issuing  permits  for  such  surveys  in  their  nations’  EEZ.  The  most  common 
measure  is  to  implement  a  “marine  mammal  watch”,  a  team  of  observers  that  scans  the  ship’s  environs 
for  signs  of  presence  of  marine  mammals  to  trigger  a  shutdown  of  the  hydroacoustic  source  when 
marine  mammals  are  entering  a  predefined  exclusion  zone. 

Marine  mammal  observers  usually  scan  the  ship’s  environs  for  whales  using  binoculars  or  the  naked 
eye.  Sightings  mostly  rely  on  spotting  a  whale’s  blow,  which  might  rise  to  a  height  of  several  meters 
but  is  visible  for  a  few  seconds  only.  Hence,  in  combination  with  the  whales’  prolonged  dives,  sighting 
opportunities  are  rare,  which,  in  addition  to  the  limited  field  of  view  and  finite  attention  span  of  human 
observers,  renders  this  method  personnel-intensive  and  difficult,  even  during  fair  weather  and  daytime. 
During  darkness  it  is  not  feasible. 

Our  long-term  goal  is  to  overcome  these  difficulties  and  to  develop  a  reliable,  automatic  whale 
detection  system  for  the  full  range  of  oceanic  environmental  conditions  (wind,  sea  surface  temperature) 
and  species.  To  this  end,  we  developed  a  ship-based  thermal  imaging  system  for  automated  marine 
mammal  detection,  consisting  of  an  actively  stabilized,  spinning  IR  camera  and  an  algorithm  that 
detects  whale  blows  on  the  basis  of  their  thermal  signature.  So  far,  this  technology  has  been  tailored  to 
and  tested  under  cold  (SST  <  10°C)  water  conditions  only,  as  this  is  where  the  technology  was 
expected  to  perform  best.  To  adapt  the  technology  to  wanner  environments  and  to  test  its  performance 
there  is  the  specific  goal  of  this  proposal. 
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OBJECTIVES 


This  project  aims  at  a  cost-efficient  test  of  the  thennal  limits  of  the  abovementioned  IR  based 
automatic  whale  detection  technology  by  attempting  to  capture  whale  spouts  during  the  northward 
humpback  whale  migration,  which  occurs  annually  rather  close  to  shore  near  North  Stradbroke  Island, 
Queensland,  Australia.  Based  on  the  collected  data,  detector/classifier  optimization  shall  be  performed 
and  ensuing  system  performance  be  tested  by  subsequent  comparisons  with  concurrent  visual 
observations. 

APPROACH 

By  obtaining  continuous  IR  video  footage  during  two  successive  northward  humpback  whale 
migrations  and  collecting  concurrent  independent  (double  blind)  visual  observations  (modified  cue 
counting),  a  data  base  will  be  created  to: 

Detennine  the  perfonnance  of  the  imaging  device  in  warm  waters; 

•  Detennine  the  perfonnance  of  the  existing  automatic  detector  in  warm  waters; 

•  Develop  optimized  auto-detection  algorithms  for  the  warm  water  realm; 

•  Detennine  the  perfonnance  of  the  optimized  detector  in  warm  waters; 

•  Generate  additional  baseline  data  on  locomotive  behavior  of  migrating  humpbacks  (providing 
the  proposed  technology  proves  technically  feasible); 

•  Generate,  for  the  first  time,  data  on  night  time  humpback  migration;  (providing  the  proposed 
technology  proves  technically  feasible); 


At  AWI,  key  individuals  for  this  project  are  Daniel  Zitterbart,  overlooking  the  system  development, 
and  Elke  Burkhardt,  overlooking  the  concunent  sighting  data  acquisition.  The  field  activities  will  be 
supported  by  Mike  Noad,  University  of  Queensland,  who  has  organized  many  whale  surveys  off  North 
Stradbroke  Island. 

WORK  COMPLETED 

The  2014  field  season  was  conducted  on  North  Stradbroke  Island,  Queensland,  Australia  between 
06.06.2014  and  20.07.2014,  with  a  primary  data  collection  period  from  14.06.2014  untill  14.07.2014. 
The  main  aim  for  the  2014  field  season  was  to  collect  IR  video  footage  of  blowing  whales,  whilst 
concurrently  recording  their  position  with  visual  observations  through  a  theodolite.  We  deployed  a 
team  of  2  (IR  technology)  plus  6  (visual  observers),  with  additional  local  support. 

The  FIRST  thennal  imager  was  operated  for  total  of  854h  (35  days),  with  783h  of  IR-data  having  been 
recorded.  Theodolite  measurements  of  whales  were  conducted  (as  part  of  different  protocols,  see  table 
below)  for  a  total  of  170h  distributed  across  this  period  during  daytime.  Depending  on  the  research 
question  under  investigation,  different  recording  protocols  were  used,  involving  2-4  observers.  MMO- 
like  observation  (i.e.  observations  conducted  as  if  for  mitigation  purposes  during  a  ship-based 
hydroacoustic  survey  using  hand  held  binoculars  and  the  unaided  eye)  were  conducted  for  about  50h 
by  an  independent  team  of  observers  on  6  days. 
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The  water  temperature  varied  between  22  and  24°C. 


RESULTS 


The  primary  goal  of  this  field  season  was  to  acquire  sufficient  data  to  be  able  to  train  the  SVM 
(support  vector  machine)  of  the  automatic  whale  detection  algorithm  under  wann  water  conditions. 
This  goal  (the  collection  of  sufficient  data)  has  been  fully  achieved.  In  a  first  screening  of  the  IR 
imagery,  we  visually  identified  2330  cues  (mostly  blows)  in  a  total  of  5 1 1  encounters  (pods).  The  high 
abundance  of  blows/whales  also  allowed  allocating  6  days  to  acquire  a  first  data  set  for  (double  blind) 
visual-MMO/IR  comparisons. 

Generally,  the  data  shows  that  within  2  km  of  the  sensor  more  than  90%  of  the  observed  pods  could 
also  be  detected  by  visual  screening  of  the  IR  footage  on  computer  displays.  On  occasion,  whales 
were  detectable  at  distances  of  up  to  10  km.  The  observers  considered  the  detectability  of  blows  in  the 
IR  to  generally  be  better  during  night  than  during  the  day.  This  is  probably  due  to  a  reduction  of  the 
overall  contrast  in  the  images  during  night  time  due  to  the  absence  of  relections  of  sunlight.  Glare,  i.e. 
the  reflection  of  the  sun  on  the  sea  surface,  is  a  problem  for  marine  mammal  observations.  The 
probability  to  detect  a  surfacing  marine  mammal  in  sea  patches  with  glare  is  practically  nil.  The  angle 
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of  sea  surface  which  is  not  observable  by  an  MMO  is  dependent!  on  the  suns  position.  IR  imaging  is 
also  impeded  by  glare,  but  to  a  much  lesser  extent.  We  found  that  while  the  visual  observations  are  not 
possible  in  an  angular  section  of  15°  -  25°,  IR  is  impeded  much  less  by  glare  with  5°  -  7°  degrees  of 
non-observable  section. 


153  33’ 45"  E 
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Figure  1.  Detected  pods  by  different  techniques.  Colored  dots  indicate  a  detection  by  the  MMO  team 
classified  as  shutdown  decision.  Full  triangles  indicate  a  detection  by  the  IR  team  classified  as 
shutdown  decision.  Open  triangles:  IR  detections  of  pods  which  were  already  detected.  Open 
squares:  detections  by  the  theodolite  team  to  obtain  ground  truth. 


IMPACT/APPLICATIONS 


The  data  acquired  so  far  indicates  that  thermographic  video  of  the  FIRST  thermal  scanner  provides 
images  of  suitable  contrast  to  be  able  to  detects  the  majority  of  whales  present  within  2  km  distance  of 
the  scanner  even  under  warm  water  conditions  by  human  screening.  The  observers  subjective 
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impression  is  that  observations  as  nighttime  are  easier  than  those  at  daytime,  due  to  less  clutter  by 
reflections  of  sunlight. 

The  results  imply  that  FIRST  thermal  scanner  can  already  be  used  in  a  “manuaf’mode  (i.e.  real  time 
monitoring  by  sailors)  to  improve  night-time  mitigation  measures.  The  large  number  of  blows 
identified  in  the  thermographic  imagery  affirms  the  feasibility  of  the  development  of  an  autodetections 
algorithm,  i.e.  is  the  ultimate  goal  of  this  project. 

TRANSITIONS 

Rheinmetall  Defence  Electronics,  Bremen,  Germany,  has  acquired  a  license  of  the  AWI-developed 
data  acquisition  software  and  current  cold-water  detection  kernel  and  is  marketing  the  entire  system 
(the  FIRST-NAVY  sensor  head  and  the  AWI-built  software  components)  under  the  acronym 
AIMMMS  (Automatic  Infrared  Marine  Mammal  Mitigation  System).  However,  with  the  current 
classification  kernel  being  only  applicable  to  cold  water  realms,  marketing  only  occurs  for  polar  and 
sub-polar  regions.  Any  further  software  developments  made  during  or  sponsored  by  ONR  through 
ETAW  do  not  fall  under  the  current  licensing  agreement. 

RELATED  PROJECTS 

We  are  not  conducting  any  related  projects  at  this  time,  nor  are  we  aware  of  any  other  project  perusing 
this  technology  being  conducted  by  our  colleagues. 

We  applied  for  funding  of  a  related  test  of  a  dual-band  IR  camera,  which  holds  the  potential  of 
providing  an  improved  S/N  ratio,  however  this  proposal  has  been  funded  due  to  budget  cuts  withing 
the  German  funding  agency  BMBF. 

PUBLICATIONS 

Note:  The  papers/reports  listed  hereinafter  describe  the  current  state  of  development  with  regard  to 
“cool  water”  deployments  and  basically  describes  the  prior  knowledge  that  is  being  entered  into  the 
ETAW  (i.e.  this)  project  by  AWI. 

Zitterbart  DP,  Kindermann  L,  Burkhardt  E,  Boebel  O  (2013)  Automatic  Round-the-Clock  Detection  of 
Whales  for  Mitigation  from  Underwater  Noise  Impacts.  PLoS  ONE  8(8):  e71217. 
doi:10.1371/journal.pone. 0071217  [published,  refereed] 

Boebel  O.,  Bombosch  A.,  Burkhardt  E.,  Cammereri  A.,  Kindennann  L.,  Richter  S.,  Zitterbart  DP. 
MAPS:  Marine  Mammal  Perimeter  Surveillance.  In:  Boebel  O.,  editor.  The  Expedition  of  the 
Research  Vessel  "Polarstem"  to  the  Antarctic  in  2012/2013  (ANT-XXIX/2).  Bremerhaven: 
Reports  on  Polar  and  Marine  Research,  [in  press] 

Burkhardt  E.,  Lanfredi  L.,  Richter  S.,  Boebel  O.,  Kindermann  L.,  Zitterbart  DP.  MAPS:  Marine 

Mammal  Perimeter  Surveillance.  In:  Lucassen  M.,  editor.  The  Expedition  of  the  Research  Vessel 
"Polarstern"  to  the  Antarctic  in  2012  (ANT-XXVIII/4).  Bremerhaven:  Reports  on  Polar  and 
Marine  Research.  652 , 89  p.  2012  . 

Richter  S,  Muller  M,  Boebel  O,  Kindermann  L,  Zitterbart  DP  (2012)  MAPS:  Marine  Mammal 

Perimeter  Surveillance.  In:  Kattner  G,  editor.  The  Expedition  of  the  Research  Vessel  "Polarstem" 


5 


to  the  Antarctic  in  2011/12  (ANT-XXVIII/2).  Bremerhaven:  Reports  on  Polar  and  Marine 
Research,  pp.  56. 

Boebel  O.,  Bombosch  A.,  Kindennann  L.,  Zitterbart  DP.  MAPS:  Marine  Mammal  Perimeter 

Surveillance  In:  Fahrbach  E,  editor.  The  Expedition  of  the  Research  Vessel  "Polarstern"  to  the 
Antarctic  in  2010/2011  (ANT-XXVII/2).  Bremerhaven:  Reports  on  Polar  and  Marine  Research, 
634,102  -  110,2011.  (PDF) 

Boebel  O.,  Kindennann  L.,  Monsees  M.,  Richter  S.,  Torres  Vega  M.,  Zitterbart  DP.  MAPS:  Marine 
Mammal  Perimeter  Surveillance  In:  Bumbke  K,  editor.  The  Expedition  of  the  Research  Vessel 
"Polarstern"  to  the  Antarctic  in  2010  (ANT-XXVIE1).  Bremerhaven:  Reports  on  Polar  and 
Marine  Research,  628,  69  -  75,  201 1.  (PDF) 

Zitterbart  DP.,  Kindennann  L.,  Boebel  O.  MAPS:  MAPS:  Marine  Mammal  Perimeter  Surveillance  In: 
Gohl  K.,  editor.  The  Expedition  of  the  Research  Vessel  "Polarstern"  to  the  Antarctic  in  2010 
(ANT-XXVI/3).  Bremerhaven:  Reports  on  Polar  and  Marine  Research,  617,1 18  -  124,  2010. 

Burkhardt  E.,  Kindennann  L.,  Zitterbart  DP.,  Reimann  A.  MAPS:  Marine  Mammal  Perimeter 
Surveillance  In:  Budeus  G.,  editor.  The  Expedition  of  the  Research  Vessel  "Polarstern"  to  the 
Antarctic  in  2010  (ARK-XXIV/1).  Bremerhaven:  Reports  on  Polar  and  Marine  Research,  612,  31- 
34,  2010. 

PATENTS 

Note:  The  patent  application  below  describes  the  cunent  state  of  technology  and  basically  describes 

the  prior  knowledge  that  is  being  entered  into  the  ETAW  (i.e.  this)  project  by  AWI. 

Zitterbart  DP,  Kindennann  L,  Boebel  O;  Method  for  automatic  real-time  monitoring  of  marine 
mammals,  US  Patent  Application  20,130,070,079,  WO  Patent  Application  2,013,037,344. 

HONORS/AWARDS/PRIZES 

Zitterbart  DP,  AWI,  „BRIESE-award  for  Marine  Science”.  Briese  Schiffahrts  GmbH  &  Co.  KG. 
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